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Definition Omnibus; adjective: "Including many things or having a variety 
of purposes or uses." SHORT USAGE: Omni. 

1 Introduction 

The EngUsh translation of Leo Tolstoy's novel War and Peace has the fol- 
lowing notable property: it contains this paragraph as a subsequence. If one 
were to write the letters and spaces that appear in the book as a string, then 
there would be a subsequence of the string that is identical to the string of 
letters and spaces in this paragraph. The full property is more general than 
that - War and Peace contains as a subsequence any possible string of up 
to 950 letters and spaces (the TgX code for this paragraph has 866 charac- 
ters). That includes valid English text such as the first 950 characters of 
President Obama's Inaugural Address, as well as a string of 950 "g"s. War 
and Peace is thus a tome that is 950-omnihus (or 950-omni) over the 27 
character alphabet {a, 6, c, . . . , 2, SPACE}. 

Of course, such a text is not at all hard to create by design. Consider 
writing the string "afecd . . . xyz " 950 times. Clearly one could then find as 
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a subsequence any possible string of length at most 950. However, it seems 
difficult to arrive upon such a string purely by chance. 

In this paper, we will study properties of /c-omni strings over an alphabet 
of size a. There are thus three researcher might pursue (i) litera- 

ture and author comparisons, disputed authorship etc. (comparative litera- 
ture); (ii) constructions (combinatorics); and (iii) behavior of random strings 
(statistics, probability). We will focus on (iii), since (i) is best left to others, 
and (ii) is trivial: The shortest string that contains all the words over an 
a letter alphabet is of length ak; simply write the alphabet k times back to 
back as done above and note that an string of length < ak — 1 necessarily 
contains a letter ^ that is represented at most k — 1 times, making the fc-string 
. . . ^ impossible to obtain subsequence. 

This paper is organized as follows. In Section 2, we place related problems 
- in which occurrences of word segments must be in a substring - in con- 
text. Section 3 explores connections between omnisequences and the coupon 
collector problem; the key link between the two is given by what we term 
the "waddle lemma." Section 4 focuses on deriving conditions under which a 
random sequence is almost never or almost always fc-omni. Additionally, we 
compute exact probabilities for a sequence to be fc-omni when its length is at 
the threshold value. Section 5 is devoted to a review of some of the deeper 
properties of coupon collection. We continue, in Section 6, by deriving a 
"zero-infinity" threshold for the expected number of missing fc-sequences, 
uncovering the fact that this threshold is not the same as that for the emer- 
gence of the omni property. A more detailed analysis is then undertaken. 
We end with potential applications in Section 7 and a list of open problems 
in Section 8. We certainly raise more questions than we answer, and invite 
the reader to dig in with gusto into some of these questions. 

2 Universal Cycles and Cover Times 

Our interest in obtaining words as embedded subsequences aside, what can 
be said if each word must occur as a consecutive string rather than a scattered 
subsequence? In this case, much is known. The theories of 

(i) Universal cycles (Knuth [21], Chung, Diaconis, and Graham [8], Hurl- 
bert [19], and a Special 2009 Issue of Discrete Mathematics [35]); and 

(ii) Cover times, i.e. waiting times until all patterns occur (Mori [25] ) 
are relevant to this question. The following beautiful result of De Bruijn, the 
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simple proof of which can be found, e.g., in West's graph theory textbook 
[31j . is the starting point of all investigations on universal cycles, also known 
as U-cycles: 

Theorem 2.1. (DeBruijn) For each a and k there exists a cyclic sequence 
of length a'^ that contains as a substring each k-letter word over [a] := 
{1,2, ... ,a} precisely once. 

The above theorem, exemplified for = 3, a = 2 by the sequence 11100010, 
asserts that a cyclic listing of k letter words on an a letter alphabet can be 
written down in the most efficient way possible for each a and k. However, 
omnibus sequences are far shorter, in the minimal case, than are U-cycles, 
since the former can be of linear length ak. 

Mori [25] has done extensive work on waiting times until all patterns 
occur as a string. Consider first the waiting time for occurrences of a single 
pattern. Feller's classic text [Hj examines this question, and the somewhat 
counterintuitive answer reveals that even the expected waiting time for a 
single pattern depends on the pattern; in the case of a binary alphabet, for 
example, the expected waiting time for HHHHHH is 2^ + 2^ + 2^ + 2'* + 
2^ + 2^ = 126, but HHTTHH occurs, on average, after just 2 + 4 + 64 = 70 
fiips. The underlying reason for this is that a pure head run of length six 
overlaps itself in six ways, but overlaps of HHTTHH with itself can only 
occur in one, two, or six places. One of Mori's [25] results on the cover times 
for all patterns is as follows: 

Theorem 2.2. Define T(a) to be the waiting time until each of the a'' pat- 
terns of length k over [a] has occurred as a run. Normalize by setting 



as k ^ oo, where F{y) = exp(— e ^) is the standard Gumbel distribution 
function. 

Results on the slickest (and shortest) way of exhibiting all patterns and 
on average case behavior can thus both be seen to be beautiful and deep. 




Then, with q = 1^4; 

' ^ 6 a+1 ' 



sup\F{Y{a)<y)~F{y)\ = 0{a-''''), 
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Generalizations to cover times on graphs have been the subject of intense 
study; see, e.g. the monographs of Aldous [5], and Aldous and Fill [1]. 

Consider another example of U-cycles and omnisequences: A universal 
cycle of length (g) of 3-subsets of {1, 2, 3, 4, 5, 6, 7, 8} exists; one example is 
given by 

1356725 6823472 3578147 8245614 5712361 2467836 7134582 4681258, 

where each block is obtained from the previous one by addition of 5 modulo 
8, and the 3-letter "word" 725, e.g., is interpreted as the 3-subset {2,5,7}. 
The above U-cycle was constructed by Hurlbert [IS] in a paper that studies 
fc-subsets of an n-set for k = 3,4,5,6, and shows how difficult the problem 
is for general k and n. Notorious in this area is the $100 question of Chung 
et al. 0: 

Conjecture U-cycles of k element subsets of[n] exist for all k if the obvious 
divisibility condition n\(2) is satisfied, and if n > n^ik) is large enough. 

If we were looking for an for an omnibus listing of fc-subsets of [n], on the 
other hand, the solution would be trivial; an obvious shortest listing would be 
12 . . .n. Once again, however, we note that the length of the shortest omni 
sequence, namely n, is shorter than that of a possible shortest representation 
as a U-cycle, which would be of length (^) . (To the best of our knowledge, 
average case behavior has not been studied in this case - where one would 
be seeking to evaluate, e.g., lE(Vr), where W is the waiting time until each 
of the fc-subsets occurs as an ordered string. Average case omni behavior, on 
the other hand will follow from the discussion in the next section.) 

3 A Twist on Coupon Collection 

THE CLASSIC COUPON COLLECTOR PROBLEM: Suppose that a "coupon 
collector" wishes to collect one of each of a toys that are found in cereal 
boxes. It is well known (see, e.g.. Feller [TIj) that she expects to collect 
¥.{W) = a{l + I + ■ ■ ■ ^) := aH{l..a) a[loga + 7 + o(l)] coupons, since the 
first purchase yields the first new toy; the expected waiting time until the 
second new toy is purchased is the mean of a geometric random variable with 
parameter which equals the third toy takes on average new pur- 
chases, and so on. In addition the variance of the waiting time in the coupon 
collector problem is given by Y{W) = (a^ Yli=i 1/"^^ ~ aH{l..a — 1)) < ^a^- 
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It turns out that the omnisequence problem is inextricably linked to the 
coupon collector problem. The following key lemma will come as no surprise 

to cognoscenti. 

Lemma 3.1. (The Basic "Waddl^ Lemma".) A sequence S is k-omni if 
and only if there exists a pairwise- disjoint collection P of completed sets of 
coupons (1-omni substrings of S) such that \P\ > k. 

Proof. Sufficiency is easy to establish. Consider necessity. Suppose there 
exist m < k pairwise disjoint 1-omni substrings of S. Let these be as "tight" 
as possible, so that the last letter in any substring is the ffist occurrence of 
that letter. Let these letters be ai, 02, . . . , ctm and let a = (aia2 . . . cimC • • • c), 
[k — m as), where c is any letter not in the string after the letter am in the 
mth string. Then a is not a subsequence of the string. Contradiction. □ 

At this juncture, it should be clear how to algorithmically find any given 
length k string in a fc-omnisequence S. One can proceed greedily: read the 
omnisequence from left to right, and when the next desired letter is found, 
record its position. The above proof shows that this algorithm will always 
yield the desired string precisely when 5* is fc-omni. 

Similarly, we can design a greedy algorithm to determine the maximum k 
for which a given string S is fc-omni. Simply read across S from left to right, 
recording each time a new 1-omni substring (complete coupon collection) is 
obtained. The total number of such substrings will be the desired k. Applying 
such an algorithm to one of several English translations of War and Peace, 
the second-named author's computer demonstrated the novel to be 950-omni 
but not 95 1-omni. 

4 Threshold Behavior and Behavior at the 
Threshold 

Consider rolling a fair die with a sides and recording the sequence of rolls 
obtained. Using the fact that we are looking at disjoint renewals of the k 
required waddles, the expected number of rolls K{Wk^a) needed before the 
recorded sequence is A;-omni on [a] equals aH{l..a)k, since the mean waiting 

^REU groups are close knit social/mathematical entities, and team members often 
develop their own vernacular. In 2008, the first named author, for no apparent reason, 
decided to call 1-omni strings (or completed sets of coupons) waddles. 
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time for a single waddle is aH{l..a). By independence, moreover, V(VFjfca) = 

v(Eti w^m) = EliV(iyi,a) = Ma'Er=i lA' - - 1)) < 

Setting = Wk,a for simplicity, we note that Y(W) = o(E{W)y, not just 
for fixed a as A; — > oo and but also in general if at least one of a, k tends to 
infinity. This is our signal that W will be tightly concentrated around its 
mean; Chebychev's inequality easily leads us to the following result: 

Theorem 4.1. Let r > be a constant, and fix a > 2, n = rk, where n, k 
are both integers. Then 

1- m/c -7 -N f 0) ^/^ < aH(l..a), or 

lim P Sequence is A; — omni = < . „ ^^)^ ' 

k^oo ^ ^ [ 1, ifr>aH{l..a) 

Proof. We provide just a proof of the second part of the result; the first 
is proved similarly. Let n = kaH(l..a) + ip(k)^/ka, where (p{k) ^ oo is 
any sequence such that ^{k) = o(\/k). In other words, n is smaller than 
(1 + £)aH{l..a) ■ k. We have 

P(not omni ) = F{W > kaH{l..a) + if{k)y/ka) 

< F{W - E{W) > ip{k)Vka) 
Y{W) 



< 



ip'^{k)ka 
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as asserted. If a — ^ oo for fixed length words, the above proof may be 
modified by letting n = kaH{l..a) + aip{a) where ^{a) oo can grow at 
an arbitrarily slow rate as long as Lp{a) = o{H{l..aj). In general, though, 
we may take n — {1 + e)aH{l..a)k as long as at least one of a,k tend to 
infinity. □ 

We now explore behavior at some threshold values of n, e.g. when n — 
aH[l..a)k + 0{1). Let P{n,k,a) denote the probability that a sequence of 
length n on an alphabet [a] is A;-omni, and let N{n, k, a) be the number of 
/c-omni sequences of length n on [a]. In the binary case, when 2i7(1..2) = 3, 
we have 

Theorem 4.2. P{3k — 1, A:, 2) = | for each k. Furthermore, for constant c, 
as k ^ oo, P{3k + c,k,2) — > |. 
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Proof. We provide a constructive count of N{n, k, 2). By Lemma 3.1, a string 
is /c-omni precisely when it contains at least k disjoint 1-omni substrings. 
For any string S = (si, S2, ■ ■ ■ , Sn), let Si,,j denote the substring (s,, . . . ,Sj). 
Given a fc-omni string 5*, let be defined as follows: iq = 0, and for 

j > 1, ij is the smallest integer such that S'j is 1-omni, and m is as 

large as possible. Now define the sequence by i'j = ij — ij-i] that is, 

each i'- gives the length of the relevant 1-omni substring of 5*. 

Now suppose i'-^^ + i'2 + . . . + i'^, = t for some fixed t. Since i'l, ^2, ■ ■ ■ ,i'k > 
2, elementary combinatorics gives that there are (*^^^^) solutions to this 
equation. For each solution (ii, 12, ... , ik), there are precisely 2'^+*^""*^ choices 
for S", since each 1-omni substring can be independently chosen to be of the 
form 11 ... 10 or 00 .. . 01, and the remaining n — t elements of S can then be 
chosen arbitrarily. Thus there are a total of 



t=2k ^ ^ 



possible fc-omnisequences of length n, and the probability that a given se- 
quence of length n is /c-omni is 



P{n.k.2) 



N{n,k,2) 



^ \ k-1 

t=2k 



n-2k 



1 

- V 



i=0 



t + k-l 

k-1 



Since J2t=Wl-~i^P'' = 2^'"^ (^ee, e.g. Gould [I?]), P{3k - l,fc,2) = |. 
On the other hand, it is not hard to see that if r is constant (or indeed if 
r = o(v^)), P{3k + r, k, 2) P{3k — 1, /c, 2) = | as A; ^ 00. This proves 
the theorem. □ 



5 Old and Recent Results on Coupon Collec- 
tion 

Our intent in this section is to provide a quick review of some classical and 
recent work on coupon collection, keeping potential applications to omnise- 
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quences in mind at all times. In Section 6, we will use the groundwork laid 
down in this section to make progress beyond Theorem 4.1. 



5.1 Approaches 

Perhaps the easiest and most natural way to view the coupon collector prob- 
lem is as an occupancy problem is which we place n balls in a urns so that 
the a" possibilities are equiprobable. This is the classical approach detailed, 
e.g., in Feller yjj, and which yields, e.g., an exact expression for pb, the prob- 
ability that exactly h of the a coupons have been collected: We first choose 
the h coupons that need to be collected and then distribute the n balls into 
the corresponding h urns so that none is empty. This yields 

Pb= , 



where 



are Stirling numbers of the second kind. We will return to this key example 
in the next section when we consider the number of missing k-words, i.e. 
words that are not found as a subsequence of a given ?7,-string. Question: Is 
there a natural way to model omnibus behavior as the successful culmination 
of a dependent urn model that starts as follows: The first k letters lead to a 
ball being tossed into one (= j of the boxes; the {k + letter enables 
one to place a ball in each of C^^^) ~ (fc) = ^ boxes, etc. 

Consider next the the waiting time approach mentioned at the beginning 
of Section 3: The waiting time Wi^a for the completion of a collection of a 
coupons is expressed as the sum of a geometric random variables with declin- 
ing success probabilities 1, (a — l)/a, (a — 2)/a, . . . 1/a. This representation 
enabled us to quickly discover the fact that 

E(W'i_„) = aH{l..a) = a(loga + 7 + o(l)) (a 00), 

can be used to compute generating function, moments, etc., and was the 
basis of the method employed by Erdos and Renyi [9J to prove the extreme 
value limit theorem 

P — <x) exp{-e"^} (a ^ 00). (1) 
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Finally, and of relevance to us, the geometric representation has been used 
(see [12] for references) to work out the asymptotics for the distribution 
of the number of missing coupons, the waiting time for the b^^ coupon, etc. 
Question: Can we model the march towards "omnibusness" as the realization 
of a sequence of dependent random variables, possibly as follows: The rth 
sequence entry generates new words, where the random variable is 
supported on {0,1,..., — {^^^) } ? (We use a simpler geometric distribution 
model in Section 5.5 to generalize (1).) 

The Poisson embedding approach is at the basis of the exposition in Al- 
dous [3], who uses a heuristic to correctly "guess" several answers - both to 
the coupon collector problem and to various generalizations such as waiting 
times until most coupons are collected; until each coupon is collected A + 1 
times (alas, this is not quite the same as having an A + 1-omnisequence!); 
until each coupon is collected once when these are not equally likely to oc- 
cur; until each subset in a class is hit; etc. Lars Hoist's important paper [18] 
shows how we may embed the placement of balls in urns (or, equivalently 
the drawing of balls from urns) into a Poisson process, so that many classical 
"quota-related" occupancy problems such as the birthday problem, coupon 
collector problem, and occupancy count problem can be recast in terms of 
order statistics from a gamma distribution. In addition, this method enables 
one to provide easier solutions to the problem of multiple coupon collection 
(A + 1 coupons of each kind) Question: Can Poisson embedding be of value 
in understanding Omnibus behavior? 

Poisson Approximation is another possibility that allows one to go beyond 
waiting time analyses. When events are rare, the probability distribution of 
their counts is often well approximated by a Poisson distribution, the depen- 
dencies between the events notwithstanding. Now, if n is large compared to 
a, then a coupon being missing would be a rare occurrence. The number 
of missing coupons, or the number of empty boxes, ought to have a Poisson 
distribution. The Stein-Chen method of Poisson approximation, as painstak- 
ingly described in the monograph by Barbour, Hoist, and Janson [7], is one 
way to quantify closeness to a Poisson distribution in an appropriate metric. 
Accordingly, we find in Chapter 6 of [7], or in the paper ^ that features many 
examples related to occupancy, that the total variation distance - between 
the distribution of the number of boxes with m > 2 or more balls (birthday 
coincidences), or the number of empty boxes (missing coupons) and appro- 
priately defined Poisson distributions - is small under a set of conditions that 
permit large expected values. Now we shall see in Section 6 that it is a rare 
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occurrence for words to not be embedded in alphabet strings. Question: Can 
the count of such words have a Poisson distribution? Or are the dependen- 
cies such that a more complicated distribution is forced upon X := Yl'j=ih' 
where the indicator variable Ij equals one iff the j^^ word cannot be found 
embedded in the n-string? 

Significant progress has been made in recent years towards a fine-resolution 
understanding of coupon collection, using methods from Analytic Combina- 
torics. The paper by Zeilberger [31] has the provocative title "How many 
singles, doubles, triples, etc. should the coupon collector expect?" that ex- 
emplifies the kinds of problems under attack from mathematicians from Paris 
to Philadelphia ([12], [IS], [Ilj, [26], [M]). It is undeniable, as we shall see in 
Sections 5.4, 5.5, and 6, that it is precisely results of this nature that will help 
one understand and establish the link between that which we know (waddle 
counts; coupons) and that which we seek to know (missing word counts). 

5.2 An Aside on Expected Values 

During Spring 2009, the senior author (AG), on sabbatical at Johns Hopkins 
University, taught a class entitled Chance and Risk to a small group of liberal 
arts students. Students found embedded messages of their own creation in 
text that they randomly generated at www . random . org. The word "omnibus" 
became part of their lexicon. They felt some pride at the realization that 
another Hopkins student (SS) was a principal player in the creation of "this 
omni research." Coupon collection was thus an important class theme, and 
we derived the fact that E(iyi^a) = aH{l..a) using nothing more than the 
fact (well-motivated if not proven) that if X ~ Geo(p), then E(X) = 1/p. 
A student asked the following question, soon after the solution for a = 3 
"gumballs" was presented: "What if there aren't the same percentage of 
red [R), blue (B), and green {G) gumballs?" The famous "Yasin's gumball 
machine" problem was thus born, in which the aforementioned probabilities 
were 1/2, 1/3, and 1/6 respectively. While attempting to prepare an answer 
key, AG realized that the only solution available in the literature for coupons 
that are present in unequal proportions appeared to be the one originally 
given by Von Schelling [32], which used inclusion-exclusion and was thus 
"not suitable" for this class. Below we offer an alternative. 

Conditioning on the order in which the three colors appear (these are 
RBG, RGB, BRG, BGR, GRB, GBR with respective probabilities |- ytJ^" 
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i'i'^'^'^)' need to find the conditional expectation of the waiting 
time given the order of the first appearance of the colors. Assume that the 
order is RBG. The waiting time is then clearly 1 + a; + 6, where x is the 
additional waiting time until the B appears. It felt initially that this waiting 
time ought to be shorter than if one were waiting for a G after an R. But 
it isn't. The conditional distribution computation reveals that in fact x — 2, 
the same as the waiting time for either B or G, given that it! appeared first. 
Thus, in this example, 

E(l^i,3) = ^•[1 + 2 + 6] + ^- [1 + 2 + 3] + ^-[1 + 1.5 + 6] + 

_L. [1 + 1.5 + 2] + ! •[1 + 1.2 + 3] + ! -[1 + 1.2 + 2] =7.3, 
and, in general we have the following 

Alternative Expression for Expected Waddletime Let halls he inde- 
pendently thrown into boxes labeled 1, 2, . . . , a so that any hox hits hox j with 
probability pj. Then the expected value of the time W — Vri,pi...,pa until all 
boxes are nonempty satisfies 

a 

?7r(l) ■ ■ ■ Qnia-l) (l + g7r(l) + ■ ■ ■ + ?7r(o-l)) 

j=l neSa 

where 

1 

^Hj) = — 

i-E.=iP-(i) 

The above expression yields an expected 1-omni time of around 2250 until 
each of the letters A through Z are randomly obtained, if we generate the 
letters according to the frequency with which they actually appear in "nor- 
mal" Enghsh text. Also, the same basic technique can be used to derive a 
direct expression for the expected collection time for (say) two copies of each 
coupon in the non uniform case. 

In the next three subsections, we collect some key results that each flesh 
out some of the ideas from Sections 5.1 and 5.2. Once again, we keep our 
ear close to the ground in the hope that we will hear something of potential 
application to the omnisequence problem. 
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5.3 Variations 

There are many variations on the basic coupon collection theme. We have 
mentioned unequal coupon probabilities and waiting times until each subset 
of coupons in a certain class is hit [3]. Adler and Ross [2] study a problem that 
continues to examine the subset theme; they allow coupons to be collected in 
certain forms of subsets - a cereal box might, for example, contain a packet 
that has pictures of six baseball catchers - and the object of interest is the 
waiting time until each coupon is in at least one collected subset. Myers 
and Wilf [26] study two simultaneous collectors. What is the chance that 
they end their collections at the same time? What is the chance that, after 
being tied for a while, one collector forges ahead, never to look back? A 
series of such intriguing connections to ballot-like problems are given in [SB] . 
May [24j considers coupon collection with quotas and unequal probabilities, 
e.g., the collection of the letters in the name "Dr. Pepper," where one must 
collect three Ps, two Es, etc., and where the letters are not found with equal 
probabilities. 

A further generalization mentioned earlier, and of particular interest to 
us, is coupon collection until A + 1 copies of each coupon are scored, A> 1. 
Note that a (minimal) collection with A + 1 copies of each coupon might 
decompose into anywhere between one and A + 1 waddles. In the case that 
A = 1, the problem goes by the name of the "double dixie cup" problem and 
was first studied in the Monthly by Newman and Shepp [29j. As mentioned 
earlier, simpler proofs of several of Newman and Shepp's results were given 
by Hoist jisj^ . who exhibited, with V = denoting the waiting time 

until v4 + 1 copies of each coupon are obtained and 

V 

V* = K*A+i = - - log a - A log log a + log A\, 

' a 

that as a ^ oo, 

(i) V*i . . . , V*^ are asymptotically independent; 

(ii) nVlA+i <u)^ exp{-e-n; and 

(iii) E(K,A+i) = a(loga + Alogloga + 7-logA! + o(l)). 

It is (iii) that we draw special attention to, since it illustrates a fact that 
holds in many cover time problems: If the first cover leads to duplication 
that is logarithmic in the size of the problem, then subsequent cover times 

^See also, Myers and Wilf, where analytic combinatorial methods are used to rederive 
many of the results in [29] . 
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are faster - generating, and in a linear fashion, only log log a "duplicates." 
Here are two further examples of this phenomenon. 

(i) Covering Designs. Let l<t<A;<?7,be integers. A collection Ax of 
fc-element subsets ("blocks") of the n element set [n] := {1, . . . ,n} is said to 
be a t — (n, k, A) covering design if each t-element set is contained in at least 
A blocks. A natural question is: What is the smallest size of Erdos and 
Spencer [lOj showed that 



while Godbole et al. [16J derived the bound 



|i|(l + log (^)+(A-l)loglog(^ 



under some mild assumptions. 

(a) t-Covering Arrays. A k x n array with entries from the g-ary alphabet 
{0, 1, . . . , g — 1} is said to form a. t — {k, n, q, A) covering array ([33]) if for 
each choice of t columns, each of the g* "words" of length t may be found 
at least A times among the rows of the selected columns. For fixed n, t, q let 
k = k{n,t,q,X) be the smallest number of rows for which at— {k,n,q,X) 
covering array exists. Better bounds are known for t = 3, g = 2, A = 1, for 
example, but general upper bounds, proved in [15] are the following {A is a 
well specified constant) 

k{n,t,q, 1) < A{t- l)logn, 

and 

k{n,t,q,X) < A{{t- l)logn+ (A - 1) log log n) . 



5.4 Coupon Counts 

The discussion at the end of Sections 5.1 and 5.3 illustrates that an important 
auxiliary variable would be counts of coupons of different types. Several 
questions may be asked. Perhaps the first is how many coupons have been 
collected precisely once at the end of a successful coupon collecting quest. 
Myers and Wilf [26] solve this problem. Among their key results is the 
fact that on average H{l..a) ^ log a coupons have been collected precisely 
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once at the end of a minimal completed coupon collection. This implies 
that roughly log a coupons need to be collected a second time by the double 
dixie cup collector; the rest have already been collected twice as part of the 
first collection. Thus, heuristically, the additional waiting time until these 
singletons turn into doubles is a Tog log a, as seen above This fact reveals a 
key difference between the waiting time until each coupon is collected k times 
and the waiting time until the sequence is fc-omni, for which the expected 
value is kaH{l..a). 

A problem inverse to that in [26] was tackled by Badus et al. [5J. The 
problem they considered was the following: How many copies are there of 
the r**^ new coupon to be collected? Extending the work in [2B], Zeilberger 
[211 gave a simpler proof of a closed form formula, first derived by Foata, 
Han, and Lass JT3\, for the generating function where Yi is 

the number of coupons that have been collected precisely i times. Another 
key contribution in [13] is the computation of the multivariate generating 
function of ^{Yi = yi, . . . ,Yr = i/r, Wi^a = w). By rephrasing the problem 
in terms of a coupon collector and his ordered infinite sequence of younger 
brothers to whom duplicates are passed on sequentially, Foata and Zeilbeger 
[14j derive results about the expected numbers of missing coupons in the 
collections of younger brothers, when p brothers have complete collections. 
A simpler proof of these results, for p = 1, is provided in [1]. 

5.5 Limit Theorems 

In this subsection, we veer the discussion back to omnisequences. We have 
seen that the normalized waiting time for the coupon collector follows asymp- 
totically a Gumbel distribution as the number of coupons gets large. A gen- 
eralization to unequal coupon probabilities is given by Neal [28], using the 
Stein-Chen method [7]. A further generalization is provided by Martinez 
[23], who proves a ratio limit theorem for the waiting time until A + 1 copies 
of h coupons are obtained. For equally likely coupons, there are a host of 
approximations for small values of a; these are of the normal, saddlepoint, 
and lognormal types, and a good summary may be found in [22]. The point 
to emphasize is that the situation is complicated even for a single waddle if 
the coupon size is small. On the other hand, if k is allowed to get large, then 

■^Notice also that the problem of how many t sets or i-letter words have been cov- 
ered precisely once in a covering design or t-covering array respectively would provide an 
extension of the work in 1261 . 
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the following result on the waiting time for a sequence to become fc-omni 
follows easily from the central limit theorem. 

Theorem 5.1. Let Wk,a be the waiting time until a sequence {Xn}^=i of 
i.i.d. letters uniformly generated from {1, 2, . . . , a} becomes k-omni. Then 

P ^ ^) ^ ^ £exp{-,.V2}* (* oo), 

where S denotes the standard deviation for the waiting time Wi^a until a 
single coupon collection is obtained. 

The above result can be used to deduce, for example, that as — cx3 
(a fixed) the probability P{kaH{l..a) + \/k,k,a) that a string of length 
kaH(l..a) + ^/k is fc-omni satisfies 

P(kaH(l..a) + VI, k, a) = P < i) - * (i) . 

where $ is the standard normal distribution function; for a = 4, we get 
S = 3.8 and a limiting value of approximately 0.603. Also, Theorem 5.1 
reveals that the probability P{kaH{l..a) + 0(1), fc, a) is asymptotically 0.5 
for all a, thus extending Theorem 4.2. 

The situation is different if k is held fixed and we allow a to tend to infinity. 
By conditioning on the a!^ orders in which letters could be generated so as 
to yield an omnibus sequence, we see that Wk^a can be written as the sum 
of ak independent geometric waiting times, with precisely k having success 
probability j/a, 1 < j < a. Thus, we recognize that 

^r Wk,a-kaloga 
\ a - 

represents the distribution function of the sum of k identical copies of the 
normalized single waddle times (Wi,a — a log a)/a. The next result follows 
easily from the Erdos-Renyi result (1): 

Theorem 5.2. 



where \1/ is the distribution function of the sum of k independent Gumbel 
variables. 

Unfortunately, the representation of as given by Nadarajah [27j is not 
amenable to easy analysis. 
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6 Missing Word Counts 



We now change our approach to the omnibus problem. Instead of considering 
only sequences that contain all possible length k strings, consider strings that 
do not necessarily attain them all. Given a sequence S of length n on [a], 
define the number of missing k-sequences of S to be the number of distinct 
/c-sequences on [a] that cannot be obtained as a subsequence of S. Denote 
this quantity by M = Mk,a = Mk,a{S), so that Mk,a{S) = ^ Ik,a{S,T) 

Te[a]*= 

where the indicator variable Ik,a{S, T) — I{T) equals 1 iff the word T is not a 
subsequence of S. The following result is critical, and is in marked contrast 
to the situation when words have to occur as strings. 

Lemma 6.1. For a sequence S on [a], the probability that a k-sequence is 
missing in S is equal to the probability that any other k-sequence is missing 
in S. 

Proof. Say S is length n, and let T = (ti,t2, ■ ■ ■ ,tk) be any word. Then T 
is missing if and only if for some < j < A; — 1 wc make "j'-fold progress" 
towards the attainment of T, i.e. the first j letters of T can be found in 
5" as a subsequence, but not the first j + 1. Let us choose the spots where 
these j letters are to appear for the first time in (") ways. Label the spots as 
ii, . . . Now the letters prior to ii cannot contain the letter ti, the letters 
in between ii and i2 must be devoid of a ^2, etc. It follows that 

— §G)G)'(^)"' 

which is merely the cumulative binomial probability B{n,k — 1,1/a). The 
above expression is dependent on only n and k, but not on what sequence 
T is. Notice that, for example, when a — 2 and T = 11 ... 1, we should 
interpret the above equation as saying that T is missing if and only if the 
sequence S contains at most {k — 1) Is. □ 

6.1 The Gap 

We now calculate the asymptotics of the expected value E(Mfc_a(5')) over all 
length n strings S, as k ^ oo and n/k — r is held constant. By linearity of 
expectation. 
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/ \ N j 1 \ n-j 



j=0 



E(M.AS)) = a^^r\ I) ^) . (2) 



J J \a J \ a 



Now for n > ak, the maximum term in the sum (2) is the one corresponding 
to j = A; — 1. This is easy to see by taking ratios of consecutive terms, and 
can be made precise by the following inequality from Barbour et al. [7]: 

Bi{n,p){0, . . . ,m-l} < —Bi{n,p){m-1}, m < np+{l-p), 

[n — Ijp — [m — 1) 



where 



Bi{n,p){A) = J2('')p'i^-Pr~'- 



This leads to 

a^Bi(n, -){A;-1} < E{Mk,a{S)) < , "',,, Bi(n, -){A;-1} < 4a'=Bi(n, -){A;-1} 
a (^1 — a a 

if, e.g., we take n > ^kaH{l..a). Thus, 

for some constant A. Applying Stirling's approximation with n = rk, we see 
that 

^ ' ' (r - 1 + o(l))^27r(r - l)k Ka'-^r - 1)'-^ j 

Now we have seen that previous asymptotic results are all couched in terms 
of alphabet sizes that grow to infinity. On the other hand, omnibus behavior 
is best appreciated for long words from a fixed size alphabet. Accordingly, 
we ask what happens to E(M) as k —>■ oo, and find from (3) that with 

(a-ir-v 

Via, r) 



and a fixed, E(M) ^ as ^ oo if D{a, r) < 1, and E(M) ^ oo (fc ^ oo) 
ii D(a,r) > 1. 
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Notice the similarity to Theorem 4.1. Holding the ratio n/k — r constant, 
we again find that there is a threshold value of r at which there is a sudden 
change in the asymptotics. However, these threshold values are not equal to 
one another. Recall, e.g., that for fc-omni strings, the threshold ratio (prior 
to which the probability of a string being A;-omni was 0, beyond which it was 
1) is 2H{1..2) = 3 for a = 2. However, again for a = 2, we can show that 
D{2,r) = 1 when r ^ 4.403. What is going on? It appears that for values of 
n between 3A: and 4.403A;, sequences are omni with high probability, and yet 
the expected number of missing sequences is huge - much like the evil two- 
valued random variable X that takes on values and with probabilities 
1 — 1/n and 1/n respectively: E(X) is large even though X equals zero 
most of the time. It appears that M is similarly not concentrated around 
its mean. Specifically, rare non-omni sequences tend to have unaccomplished 
waddles that lead to very large numbers of missing words. We return to this 
question in the next section, but for now demonstrate the fact that there is a 
neghgible "gap" when the alphabet size is large. In other words, as a — > oo, 
the difference between these threshold values grows without bound, but their 
ratio converges to one: 

Theorem 6.2. Given a, let r{a) be the real solution to D{a, r{a)) — 1. Then 

r(a) t 

05 a — > oo, „;/ ^ 1- 

' aH[l..a) 

Proof. We show that for large a, a(loga + log log a) < r(a) < a(loga + 
logloga-l- 2). Since also aH(l..a) ~ aloga, a — oo, the result will follow 
immediately via the squeeze theorem. Set r'{a) — a(loga + log log a + c) for 
a large and c constant. Then 

^ e-^»/«-eT'(a)(l + o(l)). 

Thus 

D{a,r'{a)) ~ ^-iioga+iogioga+c) . ^ . a{\oga + log logo + c) 

Thus if c = 0, D{a, r'{a)) > 1, and if c = 2, D{a, r'(a)) < 1. But D{a, r(a)) = 
1, and the result follows by monotonicity. □ 
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6.2 Understanding the Gap 

A central question is the following: How many waddles does a random se- 
quence of length n contain? We seek, in other words, to understand the level 
crossing time 

T = inf{t : Wi,a,l + Wi,a,2 + ... + Wi^a,t > u} , 

where the Wi^ajs are i.i.d. random variables with distribution equal to that 
of a single waddle-time; if r = t then the sequence is t — 1-omni (there are 
t — 1 waddles) . 

Now if there are r < k waddles, then a rather naive lower bound for the 
number „ of missing words of length k is a^~^~^, as follows. Since there 
are r waddles, let Oq be a letter not contained among the letters after the rth 
waddle is accomplished. Furthermore, let be the last letters in the 

r successfully completed coupon collections. Then we see that none of the 
words aia2 . . . 0^00X1X2 . . . x^-r-i are contained in the string, where the Xjs 
are arbitrary. Thus even y/k fewer waddles than expected would lead to at 
least a^"^ missing words. 

Let n be fixed. We invoke the basic renewal equations from Section XIII. 6 
in Feller [H], that state that the number Nn of disjoint occurrences, among 
the first n trials, of a recurrent event S with mean n and variance a^, satisfies 

E(iV„)~-; V(iV„)~^. 
/i 11-^ 

We thus see that n random keystrokes on an a letter keyboard are expected 
to contain n/aH{l..a) disjoint sets of strings that do not miss any letter, and 
that the variance of this quantity is of order nir^ /QaH^{l..a). Moreover, 
is tightly concentrated around its mean, as evidenced, e.g. by Chebychev's 
inequahty or the Azuma-Hoeffding martingale inequality ([20]) that yields, 
since altering one of the keystrokes Xi , . . . , Xn can change by at most 
one, 

> < 2exp{-AV2n}, (4) 

so that for fixed a, the number of waddles is concentrated in an interval of 
width a/ n(p{n) around its expected value - which is of order G(n) - where 
(p{n) may tend to infinity arbitrarily slowly. How then can we get significantly 
fewer waddles than expected? To fix our ideas, we recall from (3) that for 



n 



aH{l..a) 
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a = 2 we expect (27/16)'^ missing words if n = 3k, the threshold value for 
the sequence to be omni, and E{Mk,a) = (256/216)^' if n = 4k. These values 
are derived from the linearity of expectation, and provide little insight into 
what causes words to be missing, the correlations between the presence or 
absence of words, etc. Now, setting a — 2 and n — Ak in (4), we see that for 
k large enough, 

P(iV„ < k/2) < F{\N^ - 1.33k\ > 0.83k) ^ (0.916)^ (5) 

Now the actual probability of having a shortfall of 0.83k or more waddles 
is certainly smaller than that given by (5), but such a shortfall would, as 
discussed above, lead to 2^/2 

missing words - and, making believe that (5) is 
sharp, an expected value of at least {\/2 ■ 0.916))'^ ~ (1.3)''' for the number 
of missing words. Now, we know this is false (the correct expected value 
is (256/216)'^ = (1.18)'^) but we believe the above crude analysis does add 
value. 

To give a more specific example, we compute the probability that a se- 
quence of length kaH{l..a) has fewer than k — \/k waddles. By Theorem 
5.1, this converges to some constant B, and leads to the conclusion that 
E(Mfc a) > B ■ which certainly tends to infinity. 

Fleshing out the relationship between unaccomplished coupon collections 
and missing word counts clearly remains a key problem that warrants deeper 
further investigation. 

7 Applications 

We believe that omnisequences have a large number of potential applications. 
Below are some of our thoughts on the matter. 

Cryptography: Omnisequences could provide a potential method for cryp- 
tography. For example, suppose that Alice and Bob meet and exchange one- 
time pads of randomly generated letters (or even an innocuous looking copy 
of War and Peace) . The encryption process for a message then becomes to 
greedily find the position of the desired letters within the pad. For example, 
given a pad of "abfpodod. . .," the ciphertext of "food" would be "3,5,7,8." 
The decryption process simply involves reading across the pad and record- 
ing the letters that appear in the relevant positions. Notice that both the 
encryption and decryption process are exceedingly simple and require very 
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little computational resources; more complicated schema can certainly be em- 
ployed. Our results show that if we want a random pad to be able to encrypt 
any message of length k, it should have length of at least 26H{1..2Q)k ^ 100k. 
(Of course, a disadvantage of this cryptographic scheme is that only about 
1% of the letters in the pad will actually be used.) This is essentially a vari- 
ation of (or perhaps identical to) schemes that have actually been employed 
in the past. 

Randomness tests: The results of the Coupon Collector problem have 
been used to analyze the randomness of data samples, such as in Kendall 
and Babington Smith [20j. It is conceivable that the related but distinct 
results we have derived for fc-omnisequences could be apphed to randomness 
tests. 

Derivation of identities: Omnisequences are a combinatorial structure 
that provide for multiple ways of counting any one event. In the process of 
doing this research, the authors stumbled upon a number of combinatorial 
identities, some perhaps not noticed before. For example, in Lemma 6.1 it 
was shown that 

|:(:)(.-ir-. 

is equal to the total number of n-sequences not missing a word T, which can 
also be calculated as 

i=k ' 

as follows: Let the ith element of S", S'j, be the first appearance in S of the 
last letter, T^, of T, given that letters Ti, . . . , Tfc_i have appeared sequentially 
in S. Now choose the positions of the relevant terms of the subsequence in 
(*~^) ways. Consider when T,-, appears in S\ each subsequent term prior to 
Tj+i in S has a — 1 choices, namely not Tj+i. The n — i elements after Si 
have a choices. Hence 

i=k ^ ^ i=k ^ 

A second such identity can be derived by considering the total num- 
ber of (minimal) 1-omnisequences of length n on [a]. First of all, we can 
construct such a sequence in the following manner. Let 0:102 . . . Oa be a 
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permutation of a, denoting the order in which the letters first appear; the 
first letter in the omnisequence is thus ai and the last is a^. The remaining 
n — a letters can then be placed with restriction that the letters between 
ai and ttj+i may acquire any of the values ai, 0^2, . . . , a^. We note that 
this construction will always yield a distinct 1-omnisequence, and further- 
more every 1-omnisequence can be constructed in this way. Now if there are 
/j — 1 letters, li> 1, between and aj+i, then we note that we can create 
]^U-i2'2-i . . . _ ]^ya-i-i i-omnisequences. Furthermore, we had a! ways of 
creating the original permutation. Hence we calculate the total number of 
1-omnisequences of length n as a! Ylii + +1 _ =n-i l'^^^2'2"^ ■ ■ ■ (a — 1)'""^"^ = 

Alternatively, we can consider fixing the last letter of our 1-omnisequence 
(which can be done in a ways). Suppose that we want to have /i, ^2, • • • , ^a-i 
copies of each of the remaining first, second, . . ., a — 1st letters in our 1- 
omnisequence. Since the arrangement of these letters is arbitrary, we have 
that there are (^^ ^^"^^^ J sequences we can construct in this way. Again, this 
construction provides a distinct 1-omnisequence, and all 1-omnisequences of 
length n can be constructed in this manner. Thus we obtain that there are 
a'Y^i^^ +i^_i=n-i (ii /a""^/ J such omnisequences. Combining our results, we 



find that 



l\-\-...+la-\=n-\ l\-\-...-\-la-\=n-\ 



hihi ■ ■ ■ jh 



Linguistics: Note that in language, letters are not randomly distributed. 
Rather, they follow some weighted distribution (even this is, of course, a 
simplified model of language). We note that our results can be thus be 
extended to languages using a weighted version of the Coupon Collector 
problem, such as is provided by Hermann Von Schelling ^32j. Using the 
frequencies of just letters and spaces, one can calculate the expected length 
of a 1-omnisequence in English is about 2250. However, our experiments 
with various text samples have shown that this is very rarely achieved and 
the real value is probably more like 4000. In any case, this provides for some 
very interesting analysis and could conceivably be put to work checking, 
say, the degree of relationship between two languages, or testing hypotheses 
regarding disputed authorship (e.g. William Shakespeare vs. Francis Bacon, 
or Christopher Marlowe, or Edward de Vere). 

Magic and fortune-telling: Certainly omnisequences could form the back- 
bone of a magic trick. In one such scheme, the magician asks an audience 
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member to secretly compose a sentence. He or she then theatrically shows 
around a piece of text that is in fact A;-omni for large k. When the audience 
member reveals his or her sentence, the magician can then "magically" find 
the sentence encoded in the text. The remarkable fact is that if the text 
is randomly generated, it only needs to be about 100 times as long as the 
desired sentence for the magician to be successful every time. Alternatively, 
a fortune-teller could use this technique to generate any desired message in 
front of a client's eyes. 

8 Open Problems 

Questions for further investigation have been mentioned throughout the pa- 
per, but here are a few others that we consider to be central. 

(i) What is the relationship between the number of waddles in a non-omnibus 

sequence and the number of missing fc-words? 

(ii) Can we approximate the distribution C{Mk^a{S)) of missing words in an 
n string? 

(iii) What is the variance of Mk^a{S)7, 
and, last but certainly not least, 

(iv) What are the general properties of two dimensional nxn arrays over [a] 
that contain all A; x A; arrays as submatrices? (we call such arrays "omnimo- 
saics." ) 

9 Acknowledgment 

The research of all four authors was supported by NSF-REU Grant 0552730 
and conducted at ETSU during the summer of 2008. The research of God- 
bole was further supported at JHU by the Acheson J. Duncan Fund for the 
Advancement of Research in Statistics. 

References 

[1] I. Adler, S. Oren, and S. Ross (2003), "The coupon collector's problem 
revisited," J. Applied Probability 40, 513-518. 



23 



[2] I. Adler and S. Ross (2001), "The coupon subset collection problem," J. 
Applied Probability 38, 737-746. 

[3] D. Aldous (1989), Probability Approximations via the Poisson Clumping 
Heuristic, Springer Verlag, New York. 

[4] D. Aldous and J. Fill (2009), Reversible Markov Chains and Ran- 
dom Walks on Craphs, Unpublished Manuscript, available on line at 
http : //www. Stat .berkeley . edu/^aldous/RWG/book.html[ 

[5] A. Badus, A. Godbole, E. LeDell, and N. Lents (2003), "Some con- 
tributions to the coupon collector problem," Extended Abstract, 2003 
Permutation Patterns Conference, Dunedin, New Zealand. Manuscript 
in preparation. 

[6] A. Barbour and L. Hoist (1989), "Some applications of the Stein-Chen 
method for proving Poisson convergence," Advances in Applied Proba- 
bility 21, 74-90. 

[7] A. Barbour, L. Hoist, and S. Janson (1992), Poisson Approximation, 
Oxford University Press. 

[8] F. Chung, P. Diaconis, and R. Graham (1992), "Universal Cycles for 
combinatorial structures," Discrete Math. 110, 43-59. 

[9] P. Erdos and A. Renyi (1961), "On a classical problem of probability 
theory," Magy. Tud. Akad. Mat. Kutato Int. Kdzl. 6, 215-220. 

[10] P. Erdos and J. Spencer (1974), Probabilistic Methods in Combinatorics, 
Academic Press, New York. 

[11] W. Feller (1968), An Introduction to Probability Theory and its Appli- 
cations, Volume 1, John Wiley, New York. 

[12] P. Flajolet and R. Sedgewick (2009), Analytic Combinatorics, Cam- 
bridge University Press. 

[13] D. Foata, G. Han, and B. Lass (2001), "Les nombres hyperharmoniques 
et la fratrie du coUectionneur de vignettes," Sem. Lothar. Combin. 47, 
Article B47A. 



24 



[14] D. Foata and D. Zeilberger (2003), "The collector's brotherhood problem 
using the Newman-Shepp symbolic method," Algebra Universalis 49, 
387-395. 

[15] A. Godbole, D. Skipper, and R. Sunley (1996), t-covering arrays: up- 
per bounds and Poisson approximations. Combinatorics, Probability and 
Computing 5, 105-118. 

[16] A. Godbole, S. Thompson, and E. Vigoda (1996), "General upper 
bounds for covering numbers, Ars Combinatoria 42. 

[17] H.W. Gould (1972), Combinatorial Identities, Morgantown, WV. 

[18] L. Hoist (1986), "On Birthday, Collectors', Occupancy and Other Clas- 
sical Urn Problems," International Statistical Review 54, 15-27. 

[19] G. Hurlbert (1994), "On universal cycles for /c-subsets of an n-element 
set," SIAM J. Discrete Math 7, 598 - 604. 

[20] M. Kendall and B. Babington Smith (1938), "Randomness and random 
sampling numbers," J. Royal Statist. Society 101, 147-166. 

[21] D. Knuth (2005), The Art of Computer Programming, Volume 4, Fas- 
cicle 2, Pearson, NJ. 

[22] D. Kuonen (2001), Computer-Intensive Statistical Methods: Saddlepoint 
Approximations with Applications in Bootstrap and Robust Inference," 
Doctoral Dissertation, Ecole Polytechnique Federale, Lausanne. 

[23] S. Martinez (2004), "Some bounds on the coupon collector problem," 
Random Structures and Algorithms 25, 208-226. 

[24] R. May (2008), "Coupon collecting with quotas," The Electronic Journal 
of Combinatorics 15, Paper No. N31. 

[25] T. Mori (1990), "More on the waiting time till each of some given pat- 
terns occurs as a run," Can. J. Math 42, 915-932. 

[26] A. Myers and H. Will (2003), "Some new aspects of the coupon collec- 
tor's problem," SIAM J. Discrete Mathematics 17, 1-17. 



25 



[27] S. Nadarajah (2008), "Exact distribution of the linear combination of p 
Gumbel random variables," International Journal of Computer Mathe- 
matics 85, 1355-1362. 

[28] P. Neal (2008), "The generahsed coupon collector problem," J. Applied 
Probability 45, 621-629. 

[29] D. Newman and L. Shepp (1960), "The double dixie cup problem," The 
American Mathematical Monthly 67, 58-61. 

[30] J. Steele (1997), Probability Theory and Combinatorial Optimization, 
SIAM, Philadelphia. 

[31] D. West (1996), Introduction to Graph Theory, Prentice Hall, New Jer- 
sey. 

[32] H. Von Schelling (1954), "Coupon collecting for unequal probabilities," 
The American Mathematical Monthly 61, 306-311. 

[33] N. Sloane (1993), "Covering arrays and intersecting codes," Journal of 
Combinatorial Designs 1, 51-63. 

[34] D. Zeilberger (2001), "How many singles, doubles, triples, etc. should 
the coupon collector expect?" Unpublished manuscript available at 
Prof. Zeilberger's website. 

[35] Forthcoming Special Issue of Discrete Mathematics, Glenn Hurlbert and 
Brett Stevens, eds. 



26 



